An Empirical Evaluation of Bagging and BoostingRichard

An ensemble consists of a set of independently trained classiiers (such as neural networks or decision trees) whose predictions are combined when classifying novel instances. Previous research has shown that an ensemble as a whole is often more accurate than any of the single classi-ers in the ensemble. Bagging (Breiman 1996a) and Boosting (Freund & Schapire 1996) are two relatively new but popular methods for producing ensembles. In this paper we evaluate these methods using both neural networks and decision trees as our classiication algorithms. Our results clearly show two important facts. The rst is that even though Bagging almost always produces a better classiier than any of its individual component classiiers and is relatively impervious to overrtting, it does not generalize any better than a baseline neural-network ensemble method. The second is that Boosting is a powerful technique that can usually produce better ensembles than Bagging; however, it is more susceptible to noise and can quickly overrt a data set.